Method for searching multimedia using progressive histogram

ABSTRACT

The present invention relates to a method for retrieving multimedia using a histogram. The present invention configures a histogram that a relatively important bin comes at the lead of the access when accessing histogram data for retrieving the multimedia and retrieves multimedia data by using the progressive histogram. According to the present invention, there are several advantages that a high retrieval capacity can be guaranteed even though only front part in the order of transfer of the histogram is used and the efficiency and capacity of retrieval can be enhanced by using a progressive histogram optimized according to the feature at every multimedia.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a method for searchingmultimedia using a histogram.

[0003] 2. Description of the Related Art

[0004] Recently, as content-based multimedia query techniques are comingto the front, the study of multimedia features affecting a queryperformance is being made actively. Most frequently used search enginesat present use global and local color information and textureinformation for image retrieval. Among them, the color information isknown as an element mostly affecting the image retrieval. Thus, thedevelopment of more effective color features is being made, and also anattempt to develop color spaces more effective to the retrieval is beingmade.

[0005] A histogram is most widely used as color information. Thehistogram is information representative of color variance of multimediadata such as images. A bin number of the histogram is determinedaccording to how the color space is quantized. Generally, in case ofquantizing a color space more finely and representing it by many colorbins, though retrieval performance is increased, the size of thehistogram is increased, which leads to a long retrieval time. Inparticular, in case of retrieval via a network, the size of thehistogram can be an important factor according to the capacity of thenetwork. In addition, in the case that the transfer of the histogram ofdata to be retrieved is interrupted at a particular portion. Moreover,there is a need to execute approximate retrieval without using theentire histogram according to the system performance of a client or thepurpose of retrieval as well as a network environment. However, in theprior art, since the entire histogram for query images and retrievableimages was comparatively retrieved, such a need could not be satisfied.Furthermore, in the case that system performance is low, comparativeretrieval of the entire histogram makes a retrieval speed lower, while,in the case that such a need is satisfied only by approximate retrieval,unnecessary retrieval time is consumed.

SUMMARY OF THE INVENTION

[0006] The present invention relates to a multimedia retrieval using ahistogram, and discloses a histogram information structure assuring ahigh retrieval performance by using only the front parts of thehistogram by positioning relatively important information at the frontside of the histogram in the order of access in time series, and amultimedia retrieval method using this histogram.

[0007] It is, therefore, an object of the present invention to achieve ahigh retrieval performance even if query images and target images arecomparatively retrieved only by parts of the histogram.

[0008] To achieve the above object, there is provided a histogram and amultimedia retrieval method according to the present invention, whichhas a histogram structure reconfigured in the order of importance ofbins so that more important bins are positioned relatively at the frontof the histogram in the order of access in time series and which enablesretrieval using only progressive parts of the front part of a histogrambit stream.

[0009] In addition, there is provided a histogram and a multimediaretrieval method according to the present invention, which enables ahigh retrieval performance even if only parts of the histogram are usedby including important bin order information of the histogram in theinformation representing the histogram and using histogram featureinformation containing these bin order information.

[0010] In addition, there is provided a method for retrieving multimediaby configuring a progressive histogram optimized according to thefeatures of each multimedia object and using the same.

[0011] In addition, there is provided a method for retrieving multimediaby configuring a progressive histogram reordered in the order ofimportance in consideration of the features of multimedia and using theprogressive histogram reordered in the order of importance and optimizedaccording to the features of multimedia.

[0012] In addition, there is provided a method for configuring aprogressive color histogram optimized according to the features of eachmultimedia and a multimedia retrieval method using the same whichensures a high retrieval performance by using only the front parts ofthe histogram in the order of transfer of the histogram during thetransfer of the color histogram when the histogram is used as featureinformation for multimedia object retrieval.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above objects, features and advantages of the presentinvention will become more apparent from the following detaileddescription when taken in conjunction with the accompanying drawings, inwhich:

[0014]FIG. 1 is a view illustrating a histogram bin transfer stream forexplaining the present invention;

[0015]FIG. 2 is a flow chart explaining a method for reconfiguring ahistogram in accordance with a first embodiment of the presentinvention;

[0016]FIG. 3 is a flow chart explaining a method for reconfiguring ahistogram in accordance with a second embodiment of the presentinvention;

[0017]FIG. 4 is a view illustrating a HMMD color space as one example ofa color space to which the present invention is adapted;

[0018]FIG. 5 is a view explaining a 184 level quantization method viewedin a HMMD cross-section;

[0019]FIG. 6 is a view illustrating an example of reorder information ofa histogram consisting of 184 bins according to the present invention;

[0020]FIG. 7 is a view illustrating an example of performance comparisonwhen partial comparative comparison is executed according to the presentinvention;

[0021]FIG. 8 is a flow chart illustrating a method for obtaining reorderinformation of a histogram in accordance with a third embodiment of thepresent invention;

[0022]FIG. 9 is a flow chart illustrating a method for obtaining reorderinformation of a histogram in accordance with a fourth embodiment of thepresent invention; and

[0023]FIG. 10 is a view illustrating a feature information structure ofa multimedia object formed of histogram and reorder information.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0024] The following detailed description of the preferred embodimentsutilizes the color histogram only as an example for the purpose ofillustrating the method for searching multimedia data using thehistogram in accordance with the present invention. The presentinvention is equally effective when applied to other types of histogram.

[0025] In the present invention, comparative comparison using only partsof a histogram enables a high retrieval performance. Such a function isvery useful for the following two uses.

[0026] Particularly, as described above, in the case that transfer ofhistogram bit stream is interrupted by causes such as a networkenvironment during the transfer of a histogram of data, retrieval usingonly transferred parts of the histogram information ensures a highretrieval performance. For this purpose, even though retrieval isexecuted in the order of transfer of bins of the histogram, i.e., onlyby using N (N<number of bins of the entire histogram) bins by priority,a high retrieval performance is to be ensured. As a second use, a serverhaving data is determined to transfer either parts or all of thehistogram according to the use of retrieval, the environment of a clientand the environment of a network, and thus optimized retrieval can beexecuted for each case. For example, in the case that the retrieval isfor use in data retrieval of an approximately similar kind, or in thecase that the network environment is very poor, retrieval can beexecuted without overworking the network environment while ensuring aminimum retrieval performance by transferring only parts of thehistogram of data to be retrieved. On the contrary, in the case that thenetwork or computer environment is very excellent and a very accurateretrieval is desired, the entire histogram can be transferred.

[0027] In order to provide the above useful function, it is necessary tooptimize the histogram so that the retrieval performance is less loweredeven when N bins of the histogram are retrieved by priority. For thispurpose, the histogram has to be reconfigured in the structure in whichimportant bins, i.e., bins more affecting retrieval, are positioned atthe front of a bit stream. In this way, the histogram can be optimizedaccording to the purpose of the present invention by reordering of bins.

[0028] A progressive histogram is multimedia feature information, forexample, indicative of reordering bins of the color histogram in theorder of importance and transferring the reordered histogram in theorder of importance. By the progressive histogram, retrieval can beexecuted only by transferred parts even in the case that the transfer ofthe histogram of data is interrupted. In addition, a high retrievalperformance can be ensured only by using these parts of the histogram.The reason of which is that a certain degree of multimedia objectretrieval is made possible by using these parts of bins since binstransferred earlier are more important bins.

[0029] In the present invention, methods for deciding important binsinclude the following two methods.

[0030] The first method is one for statically calculating thediscrimination ability of each bin for discriminating data based on asample group of data and considering a bin having larger discriminationability as a more important bin. With respect to the decision of thediscrimination ability, the variance of values for each bin in thesample group is calculated and thus it is considered that a bin having alarger variance indicates a more important bin. If data contained in thesample group are classified into groups by similarity, it can beconsidered that, as bins have a smaller variance in the same group buthave a larger variance between data in other groups, they are moreimportant bins. Theoretically, the importance of bins can be extractedmore accurately by the latter method. However, it is not easy toclassify the sample group into groups, and thus the former method isemployed. It is verified that this method also shows a very excellentperformance by an experiment.

[0031] The second method is one for considering the importance of a binby considering the features of a color space without using a samplegroup. Generally, in the case that a color is quantized by M sections ina color space, the importance of each quantized color can be foundroughly by preliminary knowledge according to chromaticity, fineness andbrightness. Generally, a region with a lower fineness has a moreimportant effect upon retrieval as compared to an area with a higherfineness. Therefore, if a color space is divided based on its fineness,the importance of divided segments is larger than the segment with alower fineness. Hence, as bins belong to an area with a lower fineness,they are more important. Accordingly, in the reordering of bins of ahistogram, bins corresponding to areas of a lower fineness arepositioned at the front side. For example, it is assumed that thereexist two areas S1 and S2 mainly according to fineness. Assuming that S1is a fineness area lower than S2 and the ratio of importance of S1 to S2is 2:1, when segments to be positioned at the front side are selected inpriority in order to decide the bin order of the histogram, the binorder can be decided in such a manner that two one S2 is selected whenS1s are selected.

[0032] Based on the above-described concept, the features of the presentinvention will now be described.

[0033] According to the first feature of the present invention, inmultimedia retrieval using a histogram, there is provided a progressivehistogram which is reconfigured in the order of importance of bins sothat retrieval using only progressive parts at the front side is madepossible based on the order of access of the histogram, and aprogressive multimedia retrieval method for performing retrieval usingonly parts of the histogram according to the use and environment ofretrieval.

[0034] According to the second feature of the present invention, thereconfiguration of the histogram in the order of importance of binsdescribed in the first feature is considered that bins having a largervariance are more important bin by means of calculation based on thevariance of bin values for each bin calculated from a sample group.

[0035] According to the third feature of the present invention, thereconfiguration of the histogram in the order of importance of binsdescribed in the first feature is considered that bins having a smallervariance in its group but having a larger variance between other groupsare more important bin by means of calculation of the variance of eachbin in its group and the variance of each bin between other groups fromthe sample group which is classified into groups by similarity ofimages.

[0036] According to the fourth feature of the present invention, thereconfiguration of the histogram in the order of importance of binsdescribed in the first feature is configured by sampling bins meaningthe segmental regions segmented by quantization of a color space inwhich a segmental region having a low fineness is selected with highprobability and appeared in the front part of the histogram.

[0037] According to the fifth feature of the present invention, when thecolor histogram described in the fourth feature uses a color space ofHMMD configured by Hue, SUM(MAX(RGB)+MIN(RGB)), DIFF(MAX(RGB)−MIN(RGB))and the HMMD color space has the range of DIFF values of 0˜255 and thedesignated DIFF values for segmenting on the basis of the shaft of DIFFare 9,29,75,200 and the five segmental regions segmented on the basis ofthe DIFF values are S1,S2,S3,S4 and S5,respectively, the five segmentalregions are configured as 184 bins by using a color quantization methodin which SI is segmented by 8 on the basis of SUM, S2 is segmented by 4on the basis of SUM and by 8 on the basis of Hue, S3 is segmented by 4on the basis of SUM and by 12 on the basis of Hue, S4 is segmented by 4on the basis of SUM and by 12 on the basis of Hue, and S5 is segmentedby 2 on the basis of SUM and by 24 on the basis of Hue.

[0038] According to the sixth feature of the present invention, thesampling probabilities at the regions of S1,S2,S3,S4 and S5,respectively, described in the fourth feature are decided on 24:12:6:2:1in order to make the sampling probabilities differ.

[0039] According to the seventh feature of the present invention, in themultimedia retrieval system using the progressive histogram described inthe first feature, the method further comprises the step of:transferring the histogram to a client or retrieving the histogram byusing the histogram in part or whole from the front part of thehistogram according to a retrieval object and a hardware environment ofthe client for the retrieval.

[0040] According to the eighth feature of the present invention, in themultimedia retrieval system using the progressive histogram described inthe first feature, the method further comprises retrieving by only usingthe received histogram data, when the histogram data of the query datais interrupted by unexpected results in transfer.

[0041] According to the ninth feature of the present invention, in themultimedia retrieval using a histogram, the multimedia retrieval methodusing a progressive histogram includes an important bin orderinformation in a retrieving engine capable of retrieving by using aprogressive part which comes at the lead on the basis of the transferorder of histogram (bit stream), retrieves by using the partial bin ofthe histogram and the bin order information, selects and uses a binaccording to the order of importance of bins.

[0042] According to the tenth feature of the present invention, theconfiguration of the histogram in the order of importance of binsdescribed in the ninth feature is considered that bins having a largervariance are more important bin by means of calculation based on thevariance of bin values for each bin calculated from a sample group.

[0043] According to the eleventh feature of the present invention, theconfiguration of the histogram in the order of importance of binsdescribed in the ninth feature is considered that bins having a smallervariance in its group but having a larger variance between other groupsare more important bins by means of calculation of the variance of eachbin in its group and the variance of each bin between other groups fromthe sample group which is classified into groups by similarity ofimages.

[0044] According to the twelfth feature of the present invention, theconfiguration of the histogram in the order of importance of binsdescribed in the ninth feature is configured by sampling bins meaningthe segmental regions segmented by quantization of a color space inwhich a segmental region having a low fineness is selected with highprobability and appeared in the front part of the histogram.

[0045] According to the thirteenth feature of the present invention,when the histogram described in the twelfth feature uses a color spaceof HMMD configured by Hue, SUM(MAX(RGB)+MIN(RGB)),DIFF(MAX(RGB)−MIN(RGB)) and the HMMD color space has the range of DIFFvalues of 0˜255 and the designated DIFF values for segmenting on thebasis of the shaft of DIFF are 9,29,75,200 and the five segmentalregions segmented on the basis of the DIFF values are S1,S2,S3,S4 andS5,respectively, the five segmental regions are configured as 184 binsby using a color quantization method in which S1 is segmented by 8 onthe basis of SUM, S2 is segmented by 4 on the basis of SUM and by 8 onthe basis of Hue, S3 is segmented by 4 on the basis of SUM and by 12 onthe basis of Hue, S4 is segmented by 4 on the basis of SUM and by 12 onthe basis of Hue, and S5 is segmented by 2 on the basis of SUM and by 24on the basis of Hue.

[0046] According to the fourteenth feature of the present invention, thesampling probabilities at the regions of S1,S2,S3,S4 and S5,respectively, described in the twelfth feature are decided on24:12:6:2:1 in order to make the sampling probabilities differ.

[0047] According to the fifteenth feature of the present invention, inthe multimedia retrieval system using the progressive histogramdescribed in the ninth feature, the method further comprisestransferring the histogram to a client or retrieving the histogram byusing the histogram in part or whole from the front part of thehistogram according to a retrieval object and a hardware environment ofthe client for the retrieval.

[0048] According to the sixteenth feature of the present invention, inthe multimedia retrieval system using the progressive histogramdescribed in the ninth feature, the method further comprises the stepsof: reconfiguring a query histogram and a retrieval target histogramaccording to the respective bin order information; andcomparing/retrieving the histogram reconfigured that an important bincomes at the lead according to the bin order information.

[0049] According to the seventeenth feature of the present invention, inthe multimedia retrieval system using the progressive histogramdescribed in the ninth feature, the method further comprises the stepsof: selecting a query histogram and a retrieval target histogram one byone according to the bin order information; and comparing/retrieving thehistogram according to the bin order.

[0050] According to the eighteenth feature of the present invention, themultimedia retrieval is performed by using a progressive histogramoptimized according to intrinsic feature of the corresponding object atevery multimedia data and the optimization means that the importanceconfiguring the corresponding histogram is differ according to theviewpoints of comparing/retrieving.

[0051]FIG. 1 illustrates one example of the application of the presentinvention. It is assumed that, in image retrieval using query images ina network environment, the transfer of a histogram 101 corresponding tothe query images is interrupted before the entire histogram arrives, andthus the histogram is received up to P1. A client calculates similarityby using the histogram received up to P1 and using only bins at the sameposition as the histogram 102 of a target image. In this way, even ifonly parts of the histogram are used, a certain degree of retrievalperformance is ensured, for thereby enabling a more effective retrieval.

[0052]FIG. 2 illustrates a method for reconfiguring bins of a histogramgenerated by a general method in the order of importance so thatretrieval performance is not lowered drastically even if only parts ofthe histogram are used. Firstly, a sample group of various data isconfigured, and then a color histogram of data corresponding to theconfigured sample group is generated. Then, the variance of each bin ofthe generated color histogram is calculated and thereafter the histogramis reordered in the order of intensity of variance.

[0053]FIG. 3 illustrates another example of reordering bins of ahistogram generated by a general method in the order of importance.Firstly, a sample group of various data is configured and then isclassified into groups by using preliminary information on data of asimilar type. The color histogram of data corresponding to theconfigured sample group is generated, and then the variance in group forhistograms of data in the same group and the variance between groups forhistograms of data in different groups are calculated. At this time, thehistogram is reordered in the order that the variance in group is smalland the variance between groups is large.

[0054]FIG. 4 illustrates a HMMD color space for explaining a progressivecolor histogram using the HMMD color space according to an embodiment ofthe present invention. The HMMD color space is a color space of a doublecone shape. The central axis thereof is represented as SUM ([MAX(RGB)+MIN(RGB)]/2), which corresponds to brightness. Fineness is increased inthe order of center to peripheral sides of the cone, which isrepresented as DIFF(MAX(RGB)−MIN(RGB)). The angle of the cone indicatesa color, which is generally represented as Hue.

[0055]FIG. 5 illustrates an example of quantization of the HMMD colorspace explained in FIG. 4 by 184 levels. As illustrated therein, theHMMD color space is divided into 5 partial areas based on DIFF and thenis subdivided into 184 segments. These segments generate a colorhistogram formed of 184 bins.

[0056]FIG. 6 illustrates reorder information generated when the 184level color histogram described in FIG. 5 is divided into 5 partialareas based on DIFF and then the color histogram is reordered bysampling bins in each partial area in the probability ratio of24:12:6:2:1. The sampling of bins means reordering of the colorhistogram in the order of selection of 184 bins with a particularprobability. Thus, the bin firstly selected by sampling must be the mostimportant bin. Therefore, if a bin has a high probability, it has a highsampling probability by priority and thus it is more likely to bepositioned at the front in reordering. For example, two partial areashaving the lowest fineness are consists of 8 segments and 32 segmentsrespectively as shown in FIG. 5. In the embodiment of the presentinvention, the probability ratio is 24:12. Thus, one bin is sampled from2 bins in the first area, and then one bin is sampled from four bins inthe next area, for thereby sampling four bins in the first area and 8bins in the next area.

[0057]FIG. 7 illustrates changes in performance when using 16 bins, 32bins, 64 bins and 128 bins in a progressive color histogram according tothe present invention. At this time, when N bins are used, a bin valueis not represented by a decimal, but is quantized and represented by nbits for spatial efficiency. For example, the representation of a binvalue by 1 bit indicates the representation of a decimal between 0 and 1by 0 or 1 based on a particular threshold. Thus, in the case that a binvalue is represented by 1 bit and the histogram consists of 16 bins, thetotal number of bits of the histogram information used in retrieval is16. In this experiment, retrieval performance is obtained byrepresenting a bin value by 1 bit, 2 bits and 4 bits for each selectednumber of bins. In FIG. 7, a first string is indicative of a selectednumber of bins, a third string is indicative of how many bits of thehistogram information are used by representing the bin value by n bits(that is, it is indicated that the bin value is also quantized), and amiddle string is indicative of performance. The same number of bins isrepresented by 4 bits, 2 bits and 1 bit, respectively. For example, binnumber 184 is represented by 4 bits (total number of bits 736), by 2bits(total number of bits 368) and by 1 bit(total number of 184). In themiddle string, the smaller the number of bits is, the more excellent theperformance indicated by the middle string is. As illustrated in thedrawing, the representation of a bin value by only 4 bits is adequatefor achieving a high performance. The smaller the number of bins usedfor retrieval in the entire color histogram is, the lower theperformance is. However, the width of the drop is not so large.

[0058] As described above, a method for reordering bins of the histogramin the order of importance can be obtained by statically analyzing thehistogram of large quantity of multimedia data. However, there are somany various multimedia data, and thus even if a bin is so important ina particular multimedia group, it may be less important in anothermultimedia group. Thus, it is possible to achieve a higher retrievalperformance by using a histogram optimized for each multimedia data andconfigured in a different bin order based on the features of eachmultimedia data.

[0059] For reordering of bins, a statistical method may be used forcommon application regardless of the type of multimedia. However, inthis case, the respective characteristic features of each multimediacannot be reflected, and thus there is a certain limitation in theperformance. Therefore, in order to generate more optimized progressivehistogram, it is necessary to generate and use a different progressivehistogram conforming to the features of each multimedia object, forthereby achieving a retrieval optimized for each multimedia object.Here, the different progressive histogram conforming to the features isindicative of a histogram reordered in the order of importance inconsideration of the features of each multimedia. That is, it is ahistogram reordered according to the features of multimedia. Thus, inthis case, the order of bins configuring the histogram is varieddependant upon each multimedia object. In order to configure such anoptimized progressive histogram, firstly, it is necessary to calculatethe importance of each bin for comparative comparison of each multimediaobject. In this way, the importance of bins of each multimedia can becalculated by using a user feedback or using group information if thosebins have bin grouped.

[0060] Firstly, a method using a user feedback will be explained. Theuser feedback means that a user informs a system of information aboutsimilar objects and dissimilar objects based on the result of retrievalas a feedback when retrieving a similar object upon a query of acorresponding object. For example, in case of image retrieval, the userinforms the system of what images similar to the images that he or sheseeks and what images dissimilar to the images that he or she seeks arebased on the result of initial retrieval (user feedback). The systemthusly having received the user feedback can calculate the importance ofeach bin of the histogram of the corresponding object by using feedbackinformation about similarity and dissimilarity.

[0061] The method for calculating the importance w_(i) of bin i is asfollows.

w _(i) =af _(I)(i)+bf _(R)(i)

[0062] Here, a and b are constants, fI(i)=(pmi)/qvi (similarity betweensimilar images for bin i), fR(i)=pmi×qvi (dissimilarity betweendissimilar images for bin i), p and q are constants, mi is the averageof bin i values in the corresponding image group, vi is the variance ofbin i values in the corresponding image group. With respect to theabove-described importance wi of bin I, in the case that feedback imagesare only similar images or only dissimilar images, the importance ofeach element can be calculated by setting one of a and b as 0.

[0063]FIG. 8 illustrates an order of obtaining reorder information of ahistogram using a user feedback so that retrieval performance is not solowered even if only parts of the histogram are used. In this method,information about similar objects of the objects to be reordered arereceived from the user feedback, and thereafter the similarity betweensimilar objects and the corresponding bin is measured for each bin ofthe histogram. In proportion to the similarity thusly calculated, theimportance of each bins is increased. Then, when the importance of everybins is calculated for every similar objects, the reorder information ofthose bins are calculated in the order of importance of every bins.

[0064]FIG. 8 will be explained in detail. Step 201 is a step forreceiving a feedback (information about similarity or dissimilarity) forthe result of initial retrieval from a user. Step 202 is a step forobtaining similar/dissimilar multimedia object information according tothe user feedback. In step 203, a first similar/dissimilar multimediaobject is designated as a target object. In step 204, I bin is set as 0.In step 205, similarity Si is measured by comparing I-th histogram binvalues of a query object and a target object. In step 206, theimportance of I bin is increased by the similarity Si. A series of stepsfor measuring the similarity and increasing the importance of I bin bythe measured similarity are repeated until I reaches the total number ofbins in steps 207 and 208. In this way, the generation of the importanceof I bin for the corresponding object is completed. Then, if there is nonext object, histogram order information is established in the aboveorder of importance in steps 209 and 210. If there is the next object,the target object is set as the next similar/dissimilar object, and thesteps for measuring the similarity Si and generating the importance arerepeated by beginning with I bin=0 in steps 209 and 211.

[0065] Next, a method for using grouping information will be explained.For more effective multimedia retrieval, grouping information of othermultimedia objects in the same class can be included for each multimediaobject. On the other hand, multimedia objects in a particular databasecan be grouped by class. In this manner, if there is information aboutgroups of the same class, the importance of each bin can beautomatically calculated by using this information. That is, multimediaobjects in the same class are identical to the feedback informationabout similar objects from the abovedescribed user feedback. Thus, theimportance of a bin can be calculated by applying the above-describedimportance wi calculation method to the histogram of these objects inthe same way.

[0066]FIG. 9 illustrates an order of a method for using groupinginformation in place of a user feedback as another method for obtainingreorder information of a histogram. In this method, information aboutsimilar objects of the object to be reordered is received from groupinformation previously configured in step 301. The subsequent steps areidentical to the ones of the method described in FIG. 8. That is,information about similar objects of the object to be reordered arereceived from group information previously configures, and thensimilar/dissimilar object information are obtained in step 302. A firstsimilar/dissimilar multimedia object is designated as a target object instep 303. The similarity between bins of the histogram I=0 through N ismeasured and then the importance of the corresponding I bin is increasedby the similarity Si in steps 304 through 308. If there is no nextobject, histogram order information is established in the above order ofimportance in steps 309 and 310. If there is the next object, the stepsare repeated starting from 304 in steps 309 and 311.

[0067] In order to compare and retrieve the produced histogram, asdescribed above, order information representing that the bin isreordered on what importance and on what order is required. For example,calculation of similarities of two images will be described by usinghistogram. In this case, since the order of two histograms is differ,bin order information is required as feature information of thehistogram in order to compare among the bins, in which a mapping isperformed. Even though the order of the histogram is differ, it ispossible to compare among the reordered bins according to anyimportance.

[0068]FIG. 10 illustrates a feature information structure of amultimedia object formed of histogram and reorder information. A featureinformation structure 400 has a histogram information 401 as featureinformation of the multimedia and a reorder information structure 402representing that the order of each bin of the histogram is arranged onwhat importance. In here, the reorder information is a one dimensionalmatrix type in which the order of each bin of the histogram is arranged.

[0069] According to the present invention, in the multimedia retrievingusing a histogram, it is possible to perform the high retrievingcapacity even though the partial histogram is compared and retrieved.Especially, when the data transfer is interrupted in transferring thehistogram of data, high retrieving capacity is guaranteed even thoughonly the received part is retrieved.

[0070] Also, according to the present invention, there is an advantagethat a server having data discriminates whether the histogram istransmitted partially or wholly according to an object of retrieval andthe environment of a client and a network, thereby capable of performingan optimum retrieval at every case. Also, according to the presentinvention, there is an effect that a progressive histogram optimizedaccording to the feature of every object is used, so that a highretrieval capacity is possible even though the comparison and retrievalare performed by using a partial histogram. Especially, according to thepresent invention, a different feature on each object is considered tothe reorder information, thereby providing a very high capacity and anobject-oriented retrieving service.

[0071] While the invention has been shown and described with referenceto certain preferred embodiments thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims.

What is claimed is:
 1. A histogram in which a relatively important bincomes at the lead of the access on the basis of the access order of timeseries grounds in order to retrieve multimedia using the histogram. 2.The histogram according to claim 1, wherein the histogram is a colorhistogram.
 3. A progressive multimedia retrieval method comprising thesteps of; reconfiguring a histogram that a relatively important bincomes at the lead of the access when accessing histogram data forretrieving; and comparing/retrieving a query histogram and a retrievaltarget histogram according to the order of importance of bins, whenaccessing and then comparing/retrieving the histogram reconfigured asthe order of importance, the access order reconfigured as the order ofimportance.
 4. The progressive multimedia retrieval method in accordancewith claim 2, wherein the comparison and retrieval of the queryhistogram and the retrieval target histogram are performed asinformation having the meaning that the progressive part, which comes atthe lead relatively, represents the whole corresponding histogram, whenaccessing the histogram reconfigured as the order of importance,
 5. Theprogressive multimedia retrieval method in accordance with claim 2,wherein the reconfiguration of the histogram in the order of importanceof bins is considered that bins having a larger variance are moreimportant bin by means of calculation based on the variance of binvalues for each bin calculated from a sample group.
 6. The progressivemultimedia retrieval method in accordance with claim 2, wherein thereconfiguration of the histogram in the order of importance of bins isconsidered that bins having a smaller variance in its group but having alarger variance between other groups are more important bins by means ofcalculation of the variance of each bin in its group and the variance ofeach bin between other groups from the sample group which is classifiedinto groups by similarity of images.
 7. The progressive multimediaretrieval method in accordance with claim 2, wherein the reconfigurationof the histogram in the order of importance of bins is configured bysampling bins meaning the segmental regions segmented by quantization ofa color space in which a segmental region having a low fineness isselected with high probability and appeared in the front part of thehistogram.
 8. The progressive multimedia retrieval method in accordancewith claim 6, wherein, when the histogram uses a color space of HMMDconfigured by Hue, SUM(MAX(RGB)+MIN(RGB)), DIFF(MAX(RGB)−MIN(RGB)) andthe HMMD color space has the range of DIFF values of 0˜255 and thedesignated DIFF values for segmenting on the basis of the shaft of DIFFare 9,29,75,200 and the five segmental regions segmented on the basis ofthe DIFF values are SI,S2,S3,S4 and S5,respectively, the five segmentalregions are configured as 184 bins by using a color quantization methodin which S1 is segmented by 8 on the basis of SUM, S2 is segmented by 4on the basis of SUM and by 8 on the basis of Hue, S3 is segmented by 4on the basis of SUM and by 12 on the basis of Hue, S4 is segmented by 4on the basis of SUM and by 12 on the basis of Hue, and S5 is segmentedby 2 on the basis of SUM and by 24 on the basis of Hue.
 9. Theprogressive multimedia retrieval method in accordance with claim 8,wherein the sampling probabilities at the regions of S1,S2,S3,S4 and S5,respectively, are decided on 24:12:6:2:1 in order to make the samplingprobabilities differ.
 10. The progressive multimedia retrieval method inaccordance with claim 2, wherein, in the multimedia retrieval systemusing the progressive histogram, the method further comprises the stepof: transferring the histogram to a client or retrieving the histogramby using the histogram in part or whole from the front part of thehistogram according to a retrieval object and a hardware environment ofthe client for the retrieval.
 11. The progressive multimedia retrievalmethod in accordance with claim 2, wherein, in the multimedia retrievalsystem using the progressive histogram, the method further comprisesretrieving by only using the received histogram data, when the histogramdata of the query data is interrupted by unexpected results in transfer.12. The progressive multimedia retrieval method in accordance with claim2, wherein the histogram is a color histogram.
 13. A histograminformation representation structure comprising: bin order informationdeciding an order that a relatively important bin comes at the lead ofthe access in the access order of time series grounds in order to queryusing the histogram.
 14. The progressive multimedia retrieval method inaccordance with claim 13, wherein the histogram is a color histogram.15. A multimedia retrieval method using a progressive histogramcomprising the steps of: producing bin order information deciding anaccess order that a relatively important bin comes at the lead of theaccess when accessing histogram data for retrieving in the multimediaretrieval using the progressive histogram; retrieving partial bin of thehistogram by using the bin order information; and comparing/retrievingthe histogram by selecting the bin according to the order of importanceof bins.
 16. The progressive multimedia retrieval method in accordancewith claim 15, wherein the histogram is a color histogram.
 17. Themultimedia retrieval method using a progressive histogram in accordancewith claim 15, wherein, in the multimedia retrieval system using theprogressive histogram, the method further comprises transferring thehistogram to a client or retrieving the histogram by using the histogramin part or whole from the front part of the histogram according to aretrieval object and a hardware environment of the client for theretrieval.
 18. The multimedia retrieval method using a progressivehistogram in accordance with claim 15, wherein, in the multimediaretrieval system using the progressive histogram, the method furthercomprises the steps of: reconfiguring a query histogram and a retrievaltarget histogram according to the respective bin order information; andcomparing/retrieving the histogram reconfigured that an important bincomes at the lead according to the bin order information.
 19. Themultimedia retrieval method using a progressive histogram in accordancewith claim 15, wherein, in the multimedia retrieval system using theprogressive histogram, the method further comprises the steps of:selecting a query histogram and a retrieval target histogram one by oneaccording to the bin order information; and comparing/retrieving thehistogram according to the bin order.
 20. A multimedia retrieval methodusing a progressive histogram comprising the steps of: calculatingimportance of bins configuring the corresponding histogram in eachmultimedia object, in which the importance is differ according to theviewpoints of comparing/retrieving, when calculating the importance ofbins configuring histogram as feature information for retrievingmultimedia; and comparing/retrieving the histogram according to theorder of importance by using the calculated information of importanceaccording to the viewpoints of comparing/retrieving.
 21. The multimediaretrieval method using a progressive histogram in accordance with claim20, the method further comprises the step of: calculating theinformation of importance of bins according to the viewpoints ofcomparing/retrieving on the basis of user's feedback information tosimilarity or dissimilarity or on the basis of group information to asimilarity object obtained in advance.
 22. The multimedia retrievalmethod using a progressive histogram in accordance with claim 20,wherein the histogram is transmitted by the order of importance of binsand the partial histogram or the whole histogram is transmittedaccording to the situation of a system or a network.
 23. The multimediaretrieval method using a progressive histogram in accordance with claim20, wherein the histogram is a color histogram.
 24. A method forconfiguring progressive histogram information comprising the steps of:producing importance of bins configuring the histogram as featureinformation for retrieving multimedia by considering the featureaccording to the viewpoints of comparing/retrieving, in which theimportance is differ in each multimedia object; and producing andstoring the order information of bin of the histogram.
 25. The methodfor configuring progressive histogram information in accordance withclaim 24, wherein the order information of bin of the histogram is aone-dimensional matrix type arranged the order of each bin of histogram.26. The method for configuring progressive histogram information inaccordance with claim 24, wherein the histogram is a color histogram.